Tom Hardwicke

Tom Hardwicke

Research Fellow

University of Melbourne



Publications


Preprints

Hardwicke, T. E., Thibault, R. T., Clarke, B., Moodie, N., Crüwell, S., Schiavone, S. R., Handcock, S. A., Nghiem, K. A., Mody, F., Erole, T., & Vazire, S. (submitted). Prevalence of transparent research practices in psychology: A cross-sectional study of empirical articles published in 2022.
[preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Sandoval-Lentisco, A., Tortajada, M., López-Nicolás, R., López-López, J. A., Wagenmakers, E.J., Sánchez-Meca, J., & Hardwicke, T. E. (submitted). Preregistration of psychology meta-analyses: A cross-sectional study of prevalence and practice.
[preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., Schiavone, S. R., Clarke, B., & Vazire, S. (Registered Report accepted in principle). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology.
[preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T. E., Hawkins, R., Mathur, M., & Williams, R. (in press). Experimentology: An open science approach to experimental psychology methods. MIT Press.
[open access book]

Fernández, L. M., Hardwicke, T. E., & Vadillo, M. A. (preprint). Retracted papers clinging on to life: An observational study of post-retraction citations in psychology. PsyArXiv.
[preprint] [data] [analysis-code]

2024

Dulitzki, C., Crane, S. M., Hardwicke, T. E., & Ioannidis, J. P. A. (2024). Expanding the Data Ark: An attempt to make the data from highly cited social science papers publicly available. Royal Society Open Science, 11(5), 240016.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2023

Hardwicke, T. E., & Vazire, S. (2023). Transparency is now the default at Psychological Science [editorial]. Psychological Science.
[paper]

Thibault, R. T., Kovacs, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P., & Munafo, M. R. (2023). Reducing bias in secondary data analysis via an Explore and Confirm Analysis Workflow (ECAW): A proposal and survey of observational researchers. Royal Society Open Science, 10, 230568.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., & Wagenmakers, E.-J. (2023). Reducing bias, increasing transparency, and calibrating confidence with preregistration. Nature Human Behaviour, 7, 15-26.
[paper] [preprint]

Hardwicke, T. E., Salholz-Hillel, M., Malički, M., Szűcs, D., Bendixen, T., Ioannidis, J. P. A. (2023). Statistical guidance to authors at top-ranked journals across scientific disciplines. The American Statistician. 77(3), 239-247.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2022

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A. Köneke, V. E., Ioannidis, J. P. A. (2022). Post-publication critique at top-ranked journals across scientific disciplines: A cross-sectional assessment of policies and practice. Royal Society Open Science, 9, 220139.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container] [media1] [media2] [media3]

TARG Meta-Research Group & Collaborators [Thibault, R. T., Hardwicke, T. E., Clark, R. W. A., Pennington, C. R., Nilsonne, G., O’Mahony, A., Drax, K., Thompson, J., Munafò, M. R.] (2022). Discrepancy review: A feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications. Royal Society Open Science, 9, 220142.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M., Rohrer, J., Romero, F., Scheel., A., Scherer, L., Schönbrodt, F., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719-748.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017). Perspectives on Psychological Science, 17(1), 239-251.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2021

Hardwicke, T. E., Szucs D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021). Citation patterns following a strongly contradictory replication result: Four case studies from psychology. Advances in Methods and Practices in Psychological Science, 4, 1-14.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C. (2021). Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study. Royal Society Open Science, 8, 201494.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2020

Hardwicke, T. E., & Goodman, S. N. (2020). How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLOS ONE, 15, e0239598.
[paper] [preprint] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020). Calibrating the scientific ecosystem through meta-research. Annual Review of Statistics and its Application, 7, 11-37.
[paper] [preprint]

Ferrero, M., Hardwicke, T. E., Konstantinidis, E., & Vadillo, M. A. (2020). The effectiveness of refutation texts to correct misconceptions among educators. Journal of Experimental Psychology: Applied, 26, 411-421.
[paper] [preprint] [materials] [data] [analysis-code]

Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). Royal Society Open Science, 7, 190806.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2019

Hardwicke, T. E., & Ioannidis, J. P. A. (2019). Petitions in scientific argumentation: dissecting the request to retire statistical significance. European Journal of Clinical Investigation, 49, e13162.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [comment1] [comment2] [comment3]

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van ’t Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23, 815-818.
[paper] [preprint]

Hardwicke, T. E., Frank, M. C., Vazire, S., & Goodman, S. N. (2019). Should psychology journals adopt specialized statistical review? Advances in Methods and Practices in Psychological Science, 2, 240-249.
[paper] [materials] [data] [analysis-code] [reproducible-container] [media]

2018

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793–796.
[paper] [materials] [data] [analysis-code] [reproducible-container] [preprint] [comment] [comment2]

Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., Lenne, R. L., Altman, S., Long, B., & Frank, M. C. (2018). Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science, 5, 180448.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [blog1] [blog2] [comment]

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLOS ONE, 13, e0201856.
[paper] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Henry Tessler, M., Peloquin, B., & Frank, M. C. (2018). A Bayesian decision-making framework for replication. Behavioral and Brain Sciences, 41, e132.
[paper] [preprint]

Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Hofelich Mohr, A., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4, 20.
[paper] [materials]

2017

Cristea, I. A., Naudet, F., Shanks, D. R., & Hardwicke, T. E. (2017). Post-retrieval Tetris should not be likened to a ‘cognitive vaccine’. Molecular Psychiatry, 23, 1972-1973.
[paper] [pdf] [data] [analysis-code]

2016

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low cost, effective method for increasing transparency. PLOS Biology, 14, e1002456.
[paper] [preregistration] [materials] [data] [analysis-code]

Hardwicke, T. E. & Shanks, D. R. (2016). Reply to Walker and Stickgold: Proposed boundary conditions on memory reconsolidation will require empirical verification. PNAS, 113, E3993-E3994.
[paper] [pdf]

Hardwicke, T. E., Mahdi, T., & Shanks, D. R. (2016). Post-retrieval new learning does not reliably induce human memory updating via reconsolidation. PNAS, 113, 5206-5211.
[paper] [pdf] [materials] [data] [analysis-code] [comment] [media]

Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016). Selection bias, vote counting, and money priming effects: A comment on Rohrer, Pashler, and Harris (2015) and Vohs (2015). Journal of Experimental Psychology: General, 145, 655-663.
[paper] [pdf] [data] [media]

Hardwicke, T. E. (2016). Persistence and plasticity in the human memory system: An empirical investigation of the overwriting hypothesis. PhD Thesis.
[thesis] [data] [analysis-code] [osf]

Before 2016

Baker, R., Dexter, M., Hardwicke, T. E., Goldstone, A., & Kourtzi, Z. (2014). Learning to predict: Exposure to temporal sequences facilitates prediction of future events. Vision Research, 99, 124-133.
[paper] [pdf]

Ludwig, C. J. H., Farrell, S., Ellis, L. A., Hardwicke, T. E., & Gilchrist, I. D. (2012). Context-gated statistical learning and its role in visual-saccadic decisions. Journal of Experimental Psychology: General, 141, 150-169.
[paper] [pdf]

Hardwicke, T. E., Jameel, L., Jones, M., Walczak, E. J., & Magis-Weinberg, L. (2014). Only human: Scientists, systems, and suspect statistics. Opticon1826, 16, 1-12.
[paper] [pdf]

Hardwicke, T. E. (2012). Biological Psychology. Psychology Learning & Teaching, 11, 112-113.
[paper]

Tom Hardwicke

Research Fellow

University of Melbourne



Publications


Preprints

Hardwicke, T. E., Thibault, R. T., Clarke, B., Moodie, N., Crüwell, S., Schiavone, S. R., Handcock, S. A., Nghiem, K. A., Mody, F., Erole, T., & Vazire, S. (submitted). Prevalence of transparent research practices in psychology: A cross-sectional study of empirical articles published in 2022.
[preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Sandoval-Lentisco, A., Tortajada, M., López-Nicolás, R., López-López, J. A., Wagenmakers, E.J., Sánchez-Meca, J., & Hardwicke, T. E. (submitted). Preregistration of psychology meta-analyses: A cross-sectional study of prevalence and practice.
[preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., Schiavone, S. R., Clarke, B., & Vazire, S. (Registered Report accepted in principle). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary. PLOS Biology.
[preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T. E., Hawkins, R., Mathur, M., & Williams, R. (in press). Experimentology: An open science approach to experimental psychology methods. MIT Press.
[open access book]

Fernández, L. M., Hardwicke, T. E., & Vadillo, M. A. (preprint). Retracted papers clinging on to life: An observational study of post-retraction citations in psychology. PsyArXiv.
[preprint] [data] [analysis-code]

2024

Dulitzki, C., Crane, S. M., Hardwicke, T. E., & Ioannidis, J. P. A. (2024). Expanding the Data Ark: An attempt to make the data from highly cited social science papers publicly available. Royal Society Open Science, 11(5), 240016.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2023

Hardwicke, T. E., & Vazire, S. (2023). Transparency is now the default at Psychological Science [editorial]. Psychological Science.
[paper]

Thibault, R. T., Kovacs, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P., & Munafo, M. R. (2023). Reducing bias in secondary data analysis via an Explore and Confirm Analysis Workflow (ECAW): A proposal and survey of observational researchers. Royal Society Open Science, 10, 230568.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code]

Hardwicke, T. E., & Wagenmakers, E.-J. (2023). Reducing bias, increasing transparency, and calibrating confidence with preregistration. Nature Human Behaviour, 7, 15-26.
[paper] [preprint]

Hardwicke, T. E., Salholz-Hillel, M., Malički, M., Szűcs, D., Bendixen, T., Ioannidis, J. P. A. (2023). Statistical guidance to authors at top-ranked journals across scientific disciplines. The American Statistician. 77(3), 239-247.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

2022

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A. Köneke, V. E., Ioannidis, J. P. A. (2022). Post-publication critique at top-ranked journals across scientific disciplines: A cross-sectional assessment of policies and practice. Royal Society Open Science, 9, 220139.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container] [media1] [media2] [media3]

TARG Meta-Research Group & Collaborators [Thibault, R. T., Hardwicke, T. E., Clark, R. W. A., Pennington, C. R., Nilsonne, G., O’Mahony, A., Drax, K., Thompson, J., Munafò, M. R.] (2022). Discrepancy review: A feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications. Royal Society Open Science, 9, 220142.
[paper] [preprint] [preregistration] [data] [materials] [analysis-code] [reproducible-container]

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M., Rohrer, J., Romero, F., Scheel., A., Scherer, L., Schönbrodt, F., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719-748.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017). Perspectives on Psychological Science, 17(1), 239-251.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2021

Hardwicke, T. E., Szucs D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021). Citation patterns following a strongly contradictory replication result: Four case studies from psychology. Advances in Methods and Practices in Psychological Science, 4, 1-14.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., deMayo, B., Long, B., Yoon, E. J., & Frank, M. C. (2021). Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study. Royal Society Open Science, 8, 201494.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2020

Hardwicke, T. E., & Goodman, S. N. (2020). How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLOS ONE, 15, e0239598.
[paper] [preprint] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020). Calibrating the scientific ecosystem through meta-research. Annual Review of Statistics and its Application, 7, 11-37.
[paper] [preprint]

Ferrero, M., Hardwicke, T. E., Konstantinidis, E., & Vadillo, M. A. (2020). The effectiveness of refutation texts to correct misconceptions among educators. Journal of Experimental Psychology: Applied, 26, 411-421.
[paper] [preprint] [materials] [data] [analysis-code]

Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). Royal Society Open Science, 7, 190806.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container]

2019

Hardwicke, T. E., & Ioannidis, J. P. A. (2019). Petitions in scientific argumentation: dissecting the request to retire statistical significance. European Journal of Clinical Investigation, 49, e13162.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [comment1] [comment2] [comment3]

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van ’t Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23, 815-818.
[paper] [preprint]

Hardwicke, T. E., Frank, M. C., Vazire, S., & Goodman, S. N. (2019). Should psychology journals adopt specialized statistical review? Advances in Methods and Practices in Psychological Science, 2, 240-249.
[paper] [materials] [data] [analysis-code] [reproducible-container] [media]

2018

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Mapping the universe of registered reports. Nature Human Behaviour, 2, 793–796.
[paper] [materials] [data] [analysis-code] [reproducible-container] [preprint] [comment] [comment2]

Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., Lenne, R. L., Altman, S., Long, B., & Frank, M. C. (2018). Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science, 5, 180448.
[paper] [preprint] [preregistration] [materials] [data] [analysis-code] [reproducible-container] [blog1] [blog2] [comment]

Hardwicke, T. E. & Ioannidis, J. P. A. (2018). Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PLOS ONE, 13, e0201856.
[paper] [materials] [data] [analysis-code] [reproducible-container]

Hardwicke, T. E., Henry Tessler, M., Peloquin, B., & Frank, M. C. (2018). A Bayesian decision-making framework for replication. Behavioral and Brain Sciences, 41, e132.
[paper] [preprint]

Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Hofelich Mohr, A., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4, 20.
[paper] [materials]

2017

Cristea, I. A., Naudet, F., Shanks, D. R., & Hardwicke, T. E. (2017). Post-retrieval Tetris should not be likened to a ‘cognitive vaccine’. Molecular Psychiatry, 23, 1972-1973.
[paper] [pdf] [data] [analysis-code]

2016

Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low cost, effective method for increasing transparency. PLOS Biology, 14, e1002456.
[paper] [preregistration] [materials] [data] [analysis-code]

Hardwicke, T. E. & Shanks, D. R. (2016). Reply to Walker and Stickgold: Proposed boundary conditions on memory reconsolidation will require empirical verification. PNAS, 113, E3993-E3994.
[paper] [pdf]

Hardwicke, T. E., Mahdi, T., & Shanks, D. R. (2016). Post-retrieval new learning does not reliably induce human memory updating via reconsolidation. PNAS, 113, 5206-5211.
[paper] [pdf] [materials] [data] [analysis-code] [comment] [media]

Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016). Selection bias, vote counting, and money priming effects: A comment on Rohrer, Pashler, and Harris (2015) and Vohs (2015). Journal of Experimental Psychology: General, 145, 655-663.
[paper] [pdf] [data] [media]

Hardwicke, T. E. (2016). Persistence and plasticity in the human memory system: An empirical investigation of the overwriting hypothesis. PhD Thesis.
[thesis] [data] [analysis-code] [osf]

Before 2016

Baker, R., Dexter, M., Hardwicke, T. E., Goldstone, A., & Kourtzi, Z. (2014). Learning to predict: Exposure to temporal sequences facilitates prediction of future events. Vision Research, 99, 124-133.
[paper] [pdf]

Ludwig, C. J. H., Farrell, S., Ellis, L. A., Hardwicke, T. E., & Gilchrist, I. D. (2012). Context-gated statistical learning and its role in visual-saccadic decisions. Journal of Experimental Psychology: General, 141, 150-169.
[paper] [pdf]

Hardwicke, T. E., Jameel, L., Jones, M., Walczak, E. J., & Magis-Weinberg, L. (2014). Only human: Scientists, systems, and suspect statistics. Opticon1826, 16, 1-12.
[paper] [pdf]

Hardwicke, T. E. (2012). Biological Psychology. Psychology Learning & Teaching, 11, 112-113.
[paper]